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Abstract 

In this paper, we study optimal control problems associated with a scalar hyperbolic con- 
servation law modeling the development of ovarian follicles. Changes in the age and maturity 
of follicular cells are described by a 2D conservation law, where the control terms act on the 
velocities. The control problem consists in optimizing the follicular cell resources so that the 
follicular maturity reaches a maximal value in fixed time. Using an approximation method, we 
prove necessary optimality conditions in the form of Pontryagin Maximum Principle. Then we 
derive the optimal strategy and show that there exists at least one optimal bang-bang control 
with one single switching time. 
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1 Introduction 

This work is motivated by natural control problems arising in reproductive physiology. The 
development of ovarian fohicles is a crucial process for reproduction in mammals, as its biological 
meaning is to free fertilizable oocyte(s) at the time of ovulation. Ovarian follicles are spheroidal, 
tissular structures sheltering the oocyte. During each ovarian cycle, numerous follicles are in 
competition for their survival. Few follicles reach an ovulatory size, since most of them undergo a 
degeneration process, known as atresia (see for instance [26]). The follicular cell population consists 
of proliferating, differentiated and apoptotic cells, and the fate of a follicle is determined by the 
changes occurring in its cell population in response to an hormonal control originating from the 
pituitary gland. 

A mathematical model, using both multi-scale modeling and control theory concepts, has been 
designed to describe the follicle selection process on a cellular basis (see [l3]). The cell population 
dynamics is ruled by a conservation law, which describes the changes in the distribution of cell age 
and maturity. 

Cells are characterized by their position within or outside the cell cycle and by their sensitivity 
to the follicle stimulating hormone (FSH). This leads one to distinguish 3 cellular phases. Phase 1 
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and 2 correspond to the proliferation phases and Phase 3 corresponds to the differentiation phase, 
after the cehs have exited the cell cycle. 

The cell population in a follicle / is represented by cell density functions p^- y.{t,x,y) defined 

on each cellular phase QJ^, where j = 1, 2, 3 denotes Phase 1, Phase 2 and Phase 3, k = 1, 2, • • • 
denotes the number of the successive cell cycles (see figure [T]) . The cell density functions satisfy 
the following conservation laws: 
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Figure 1: Cellular phases on the age-maturity plane for each follicle /. The domain consists of 
the sequence of k = 1, 2, • • • cell cycles. The variable x denotes the age of the cell and y denotes 
its maturity. The number is the threshold value at which cell cycle exit occurs and y^ is the 
maximal maturity. The top of the domain corresponds to the differentiation phase and the bottom 
to the proliferation phase. 

Let us define 

^f(t)-=Y.Y. / ypik{t,x,y)dxdy (1.2) 

j=i k=i •'^ •'^ 

as the maturity on the follicle scale, and 

M{t):=Y,Mf{t) (1.3) 

/ 

as the maturity on the ovarian scale. 

Two acting controls, Uf{t,Mf) and U{t,M) can be distinguished (see [12]). The global control 
U{t, M) results from the ovarian feedback onto the pituitary gland and impacts the secretion of 
FSH. The feedback is responsible for reducing FSH release, leading to the degeneration of all but 
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those follicles selected for ovulation. The local control Uf{t,Mf) is specific to each follicle and 
accounts for the modulation in FSH bioavailability related to follicular vascularization. 

In Phase 1 and 3, both a global control U and a local control Uf act on the velocities of aging (gf 
function, locally controlled in Phase 1), and maturation (hf function, locally controlled in Phase 1 
and 3) as well as on the loss term (A apoptosis rate, globally controlled in Phase 1 and 3). Phase 
2 is uncontrolled {gf = Tgj, /i/ = A = 0) and corresponds to completion of mitosis after a pure 
delay in age a2 — oi (no cells leave the cycle here). The increase in the cell mass occurs through 
mitosis. At the time of mitosis, a mother cell gives birth to two daughter cells, which results in a 
local doubling of the flux. Accordingly, the boundary between different cell cycles k = 1,2, ■ ■ ■ is 
expressed as (see figure [T]) 

9f{uf)p(,.^^{t,ka2,y) = 2Tgfp{^^{t,ka2,y), {t,y) G [0,r] x [0,?/^]. 

One can refer to [29] for more details on the model. 

The aging velocity controls the duration of the cell division cycle. Once the cell age has reached 
a critical age, the mitosis event is triggered and the two daughter cells enter a new cell cycle. Hence, 
there are local singularities in the subpart of the domain where y ^ ys, that correspond to the flux 
doubling due to the successive mitosis events. The maturation velocity controls the time needed 
to reach a threshold maturity ys, when the cell exits the division cycle definitively. After the exit 
time, the cell is no more able to contribute to the increase in the follicular cell mass. 

Ovulation is triggered when the ovarian maturity reaches a threshold value Ms- The stopping 
time Tg is defined as 

Ts:=mm{T\M{T)=Ms}, (1.4) 

and corresponds on the biological ground to the triggering of a massive secretion of the hypothalamic 
gonadotropin releasing hormone (GnRH). 



As a whole, system (1.1)-(1.3) combined with stopping condition (1.4) defines a multiscale 
reachability problem. It can be associated to an optimal control problem that consists in minimizing 
Tg for a given target maturity Mg. 

The follicles are then sorted according to their individual maturity. The ovulatory follicles are 
those whose maturity at time Tg has overpassed a threshold Mgi such as Mgi < Mg. The ovulation 
rate can then be computed as 

iV,,,i = Card{f I Mf{Tg) ^ M,i} . (1.5) 

The hormonal control exerted by FSH acts directly on follicular cells to control their commitment 
towards either proliferation or differentiation. In turn, the hormonal feedback exerted by the ovary 
on the hypothalamo-pituitary axis ensues from the weighted contribution of all cells distributed 
amongst all follicles. Hence, a hormone-driven competition process, occurring within the population 
of simultaneously developing follicles, is intertwined with each cell dynamics process taking place 
within a given follicle. 

A concept central to the understanding of these entangled processes is that of the management 
of follicular cell resources, both on the follicular (intensity of selection) and ovarian (triggering and 
chronology of ovulation) scales. There is indeed a finely tuned balance between the production of 
new cells through proliferation, that increases the whole cell mass, and the maturation of cells, that 
increases their contribution to hormone secretion. 

This concept has already been investigated on a mathematical ground. In [12], the authors 
studied the characteristics associated with a follicle as an open-loop control problem. They de- 
scribed the sets of microscopic initial conditions compatible with either ovulation or atresia in the 
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framework of backwards reachable set theory. Since these sets were largely overlapping, their results 
illustrate the prominent impact of cell dynamics control in the model. In [27], the author focused 
on the issue of the selection process in a game theory approach, where one follicle plays against all 
the other ones. Whether the follicle becomes atretic (doomed) or ovulatory (saved) depends on the 
follicular cell mass reached at the time when all cells stop proliferating. 

In this paper, we study an optimal control problem of follicular cell resources, where the 
control acts directly on the velocity terms. The controllability of nonlinear hyperbolic equa- 
tions (or systems) have been widely studied for a long time; for the ID case, see, for instance 
[ZlEllIIllISKISlliaillEaEl for smooth solutions and [H El O [50] for BV entropic solutions. 
In particular, provides a comprehensive survey of controllability of partial differential equations 
including nonlinear hyperbolic systems. As far as optimal control problems for hyperbolic systems 
are concerned, one can refer to |16l [T71 [T8t I30j . However, most of these monographs study the case 
where the controls are either applied inside the domain or on the boundary. Our control problem is 
quite different from the problems already studied in the literature, since the control terms appear 
in the flux. To solve the problem, we make use both of analytical methods based on Pontryagin 
Maximum Principle (PMP) and numerical computations. 

The paper is organized as follows. In section [2j we set the optimal control problem, together 
with our simplified assumptions, and we enunciate the main result. In section [3j we give some 
optimal results in the case where Dirac masses are used as a rough approximation of the density. 
We show that for finite Dirac masses, every measurable optimal control is a bang-bang control 
with one single switching time. In addition to the theoretical results, we give some numerical 
illustrations. In section |4j we go back to the original PDE formulation of the model, and we show 
that there exists at least one optimal bang-bang control with one single switching time. 

2 Problem statement and introductory results 

2.1 Optimal control problem 

We consider the following conservation law on a fixed time horizon: 



Pt + Px + ((a(y) + b{y)u)p) 



y 





where 



«(y) 




Ky) '■= ciy + C2, 




and 




ify G [0,y,), 
if y G [ys,oo) 



(2.3) 



with Us, Cs, ci and C2 being given strictly positive constants. We assume that 



< 1. 



(2.4) 



Let us denote by w a positive constant such that 




(2.5) 
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From (2.2) and (2.5), we have 

a{y) + b{y)u > 0, Vy G [0, y,], Vn G [w, 1]. (2.6) 
Throughout this paper the control u is assumed to satisfy the constraint 

uG[w,l]. (2.7) 

Let us define the exit time to as 

y{k) = ys, (2.8) 

where y{t) is the solution to the Cauchy problem 

y = a{y) + b{y)w, y{to) = 0. (2.9) 



Let us point out that, by (2.6), there exists one and only one to satisfying (2.8). Note that it is 
not guaranteed that the exit time to occurs before the final time ti, so that we may have to > ti. 
When t > to, all the cells are in Phase 3, i.e. their maturity is larger than the threshold ys- After 



time to the mass will not increase any more due to (2.3). 
We assume that there is no outer influx, i.e. 

p{t,0,y) = p{t,x, 0) = 0, yt e {to,ti), X > 0, y > 0. (2.10) 

The initial condition is 

P(0, X, y) = po{x, y), X ^ 0, y ^ 0, (2.11) 

where po is a positive Borel measure on M x M with a compact support included in [0, 1] x [0, ys]- 
For any admissible control u G L°°{{to,ti); [w, 1]), we define the cost function 

/>+oo /•+00 

J{u) :=- / ydp{tux,y), (2.12) 

Jo Jo 

and we want to study the following optimal control problem: 

minimize J{u) for u G L~((to,ti); [w,l])- (2.13) 

A similar minimal time problem was investigated in a much simpler ODE framework [6] , where 
the proliferating and differentiated cells were respectively pooled in a proliferating and a differenti- 
ated compartment. The author proved by PMP that the optimal strategy is a bang-bang control, 
which consists in applying permanently the minimal apoptosis rate and in switching once the cell 
cycle exit rate from its minimal bound to its maximal one. 

In contrast, due to the fact that c is discontinuous, we cannot apply PMP directly here. The 
idea is to first consider optimal control problems for Dirac masses (see section [s]), and then to pass 
to the limit to get optimal control results for the PDE case (see section |4]) . For "discontinuous" 
optimal control problems of finite dimension, one cannot derive necessary optimality conditions by 
applying directly the standard apparatus of the theory of extremal problems [U \2T\ I28j . The first 
problem where the cost function was an integral functional with discontinuous integrand was dealt 
in |2]- Later, in [31], the author studied the case of a more general functional that includes both the 
discontinuous characteristic function and continuous terms. There, the author used approximation 
methods to prove necessary optimality conditions in the form of PMP. One of the difficulties of our 
problem is that both the integrand of the cost function and the dynamics are discontinuous. 

The main result of this paper is the following theorem. 



5 



Theorem 1. Let us assume that 



h > to, (2.14) 

2ys — ci > ana Cs > . (2.15) 

Vs 

Then, among all admissible controls u G L°°((to,ii); [w, 1]), there exists an optimal control for 



the minimization problem (2.13| such that 



G [to,ii] such that = w in {to,t^:) and it* = 1 in (t=i,,ti). (2-16) 

2.2 Simplifications with respect to the original model 

To make the initial problem tractable, we have made several simplifications on the model 
dynamics. 

^i. We consider only one developing follicle, i.e. / = 1; 
82- There is no loss term anymore, i.e. A = 0; 
^3. The age velocity is uncontrolled, i.e. gf = 1; 



S4. The cell division is represented by a new gain term, i.e. c{y) defined by (2.3); 
5*5. The target maturity Mg can always be reached in finite time. 

Simplification (^i) means that, in this problem, we are specially interested in the coupling 
between the condition needed to trigger the ovulation, on the one hand, and the control of the 
follicular cell dynamics, on the other hand. We consider this interaction independently of the 
process of follicle selection, in the sense that we isolate the dynamics of one specific follicle, as if 
we could ignore the influence of the other growing follicles. 

(5*2) to (5*4) allow us to simplify the cell dynamics. In (5*2), we neglect the cell death, which 
is quite natural when considering only ovulatory trajectories, while, in {S3), we consider that the 
cell age evolves as time (so that the cell duration is constant and uncontrolled). Moreover, the 
cell division process is distributed over ages with (6*4), so that there is a new gain term in the 
model instead of the former mitosis transfer condition. Even if the way that cell proliferation is 
represented is less realistic (no real mitosis), this does not impact the dynamics too much as long 
as the average cell cycle duration is preserved. 

Even if it is simplified, the problem studied here still captures the essential question of the com- 
promise between proliferation and differentiation that characterizes terminal follicular development. 
On the one hand, the follicle can benefit from a strong and quick enlargement of its cell popula- 
tion. On the other hand, this enlargement occurs at the expense of the maturation of individual 
cells. This compromise was instanced here as a problem of composition of velocities. We chose as 
control variable the nonlocal variable Uf, which is the most downstream input representing FSH 
on the follicular level in the original model, since it intervenes directly in the aging and maturation 
velocities. A relatively high aging velocity tends to favor cell mass production while a relatively 
high maturation velocity tends to favor an increase in the average cell maturity. 



As shown in section 2.4, assumptions {S2) and (55) allow us to replace a minimal time criterion 
by a criterion that consists in maximizing the final maturity. Hence, from the initial, minimal time 
criterion, we have shifted, for sake of technical simplicity, to an equivalent problem where the final 
time is fixed and the optimality criterion is the follicular maturity at final time. On the biological 
ground, this means that for any chosen final time ti, the resulting maturity at final time Mj(ti) can 
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be chosen in turn as a maturity target which would be reached in minimal time at time ti. It can 
be noticed that in the initial problem (1.4), there might be no optimal solution without assumption 
(55), if the target maturity is higher than the maximal asymptotic reachable maturity. 



2.3 Solution to Cauchy problem iKll, ^AO^ and (|2ll| 



In this section, we give the definition of a (weak) solution to the Cauchy problem 

Pt + Px + {{a{y) + b{y)u)p)y = c{y)p, t G (to, ^i), x > 0, y > 0, 
< pit,0,y) = p{t,x,0) = 0, t£{to,ti),x>0,y>0, 
, P(0, X, y) = pq{x, y), x > 0, y > 0, 



(2.17) 



where po : [0, 1] x [0, ys] — [0, +00) is given. 

Let us denote by y the asymptotic maturity, i.e. the positive root y of a(y) + b{y)u = with 
control u = 1. From (2.2), we have 



Cl + + 4C2 



(2.18) 



Let u G L°^((to,ti); 1])- Let us define the map 



^ ■.[to,h]x[0,ys]x L^{ito,hy,[w,l]) ^ [0,y] 

{t,yo,u) '^{t,yo,u) 



by requiring 



'dt 



{t, yo,u) = a{^{t, yo, u)) + b{^{t, yo, u))u{t), 



[ ^(^0,2/0,-") = yo- 
Let Po be a Borel measure on M x M such that 



Po ^ 0, 

and the support of po is included in [0, 1] x [0,ys]- 



(2.19) 



(2.20) 
(2.21) 



Let K := [0,ti — to + 1] x [0,y]. Let M{K) be the set of Borel measures on K, i.e. the set of 
continuous linear maps from C^{K) into M. The solution to Cauchy problem (2.17) is the function 
p : [to,ti] M{K) such that, for every (/? G C°(/C), 



^(a, P)dp{t, a,P)= [[ ifixo + t-to, ^{t, yo, u)) ^(*(^.2^o,n))ds ^^^^^^^ ^2.22) 



K 



K 



We take expression (2.22) as a definition. This expression is also justified by the fact that if po is 
a L°° function, one recovers the usual notion of weak solutions to Cauchy problem (2.17) studied 
in [51 Uni [291 EO] ; as well as by the characteristics method used to solve hyperbolic equations (see 
figure [2]) . 
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Figure 2: Construction of a weak solution by backward tracking of the characteristics. The variables 
X and y respectively denote the age and the maturity, is the threshold maturity and y is the 
asymptotic maturity. The initial mass concentrates in the shaded area [0,1] x [0,2/^]- The curve 
^ = (x, ?/) is the characteristic curve passing through (t,a, P) that intersects the initial plane t = to 
at {to,xo,yo). 



2.4 Minimal time versus maximal maturity 



In this section, we show that the two optimal control problems enunciate either as: "mini- 
mize the time to achieve a given maturity" or "achieve a maximal maturity at a given time" are 
equivalent. 

Let po be a nonzero Borel measure on M x R satisfying (2.20) and ( |2.21 ). Let us denote by 
M'^{t) the maturity at time t for the control u G L°°{(tQ, ti); [w, 1]) (and the initial data po)- 
A. For fixed target time ti, suppose that the maximum of the maturity 



M 



(2.23) 



is achieved with an optimal control u G L°°((to, ti)',['w,l]). Then we conclude that for this fixed M, 
the minimal time needed to reach M is ti with the same control u. We prove it by contradiction. 
We assume that there exists another control u € L°°((to,ii); [w, 1]) such that 



M^{ii) = M, ii<ti. 
We extend u to [to;ii] by requiring {t = 1 in {ti,ti]. Let us prove that 

t G — 5- M'"'{t) is strictly increasing. 



(2.24) 
(2.25) 



Let p : [to,ti] — )• M{K) be the solution to the Cauchy problem (see section 2.3) 

Pt + Px + {{a{y) + h{y)u)p)y = c{y)p, t G (to, ^i), x > 0, y > 0, 
p{t, 0, y) = p{t, X, 0) = 0, t£ (to, h), X > 0, y > 0, 
P(0, X, y) = po(x, y), x > 0, y > 0. 

Note that a{y) + b{y) > for every y G [0, y) and that, for every t G [to,ti], the support of p{t) is 
included in [0,ti — to + 1] x [0,^^). Together with (2.22) for p = p and (^(a,/3) = (3, this proves 
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(2.25). From (2.25) it follows that 



M"(ti) > M"(ti) = M, (2.26) 

which is a contradiction with the optimality of u. 

B. For any fixed target maturity M, suppose that the minimal time needed to reach M is ti with 
control u S L°°((to,ii); [w, 1]). Then we conclude that for this fixed target time ti, the maximal 
maturity at time ti is M with the same control u. We prove it again by contradiction. We assume 
that there exists another control u G L°°((to,ii); [w, 1]) such that 

M"(ti) > M. (2.27) 

Then by the continuity of M^(t) with respect to time t, there exists a time ti < ti such that 

M"(ti) = M, (2.28) 

which is a contradiction with minimal property of ti . This concludes the proof of the equivalence 
between the two optimal control problems. ■ 



3 Optimal results for finite Dirac masses 



In this section, we give results on the optimal control problem (2.13) when the initial data 
Po ^ is a linear combination of a finite number of Dirac masses. For {a, (3)^'^ gM?, we denote by 
6a, 13 the Dirac mass at (a, ^Y'^. We assume that, for some positive integer A^, there exist a sequence 
{{xi^,X2^))i^^^i fyfj of elements in [0, 1] x [0,ys\ and a sequence {x'^^)k£{i,...,N} of strictly positive 
real numbers such that 



N 

k=l 

We consider the following Cauchy problem: 



X 



X' 



kO 



where 



/(x^n)= a(:E^) + 6(x^) n , 



C\Xrt I OCo 



It is easy to check that the maximal solution to Cauchy problem (3.2) is defined on [tQ,t 




(3.1) 



(3.2) 



(3.3) 



tij. 



One can also easily check that the solution to Cauchy problem (2.17), as defined in section 2.3 



IS 



N 



pit) = X]^3(^)'^x^(^),xt(^)■ 



fc=l 



The cost function J defined in (2.12) now becomes 

N 

Y,-4ih)xl{h 



J{u) 



(3.4) 



(3.5) 



k=l 



One of the goals of this section is to prove that there exists an optimal control for this optimal 



control problem and that, if (2.14) and (2.15) hold, every optimal control is bang-bang with only 



one switching time. More precisely, we prove the following two theorems. 
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Theorem 2. The optimal control problem (2.13) has a solution, i.e., there exists S 
i°°((io,ii); [wA]) such that 

Jiu^) = inf Jin). 

«GL°°((to,ii);[«;,l]) 



Theorem 3. Let us assume that (2.14) and (2.15) hold. Then, for every optimal control u^, for 
the optimal control problem (2.13), there exists G (ioi^i) such that 

u^: = w in (to>i*) O'lT'd u* = 1 m (3-6) 

This section is organized as follows. In subsection 3.1 we prove Theorem [2j In subsection |3.2| 
we prove a PMP (Theorem |4]) for our optimal control problem. In subsection |3.3| we show how to 
deduce Theorem [3] from Theorem |4l 

3.1 Proof of Theorem [2] 

For sake of simplicity, we give the proof of Theorem [2] only in the case where = 1. The case 
where A'^ > 1 can be treated similarly, except that the notations are more complicated. To simplify 
the notations we also delete the k = 1 index. 

Let (M")neN C L^{{to,ti); [w, 1]) be a minimizing sequence of the functional J: 

lim J(u") = inf J(n). 

n^oo ueL°°{{to,ti);lw,l]) 



We have, using (3.2), (3.3) and (3.5), 



J(n") 



to 



where and are solutions to the Cauchy problem 



i;^ = a(x5) + 6(x^K, x^(to) = x^, 
i;^ = c(x5)x^, x^{to) = xl. 



(3.7) 



(3.8) 
(3.9) 



Since (n'^)„gN are in L'^{{tQ,ti);[w,l]), there exists G L°°((to, ti); [w^, 1]) and a subsequence 
(M"'=)fegN such that u"* ^ ti^, in L°°{to,ti) as k ^ +oo. For sake of simplicity, we still denote the 
subsequence by (u")„gN- 

Let X2 and X3 be the solutions to the Cauchy problem 



X2 = a{x2) + b{x2)u^, X2{to) = X2, 

X3 = C{X2)X3, X3{to) = X^. 



Similarly to (3.7), one has 

J(n*) 



to 



(a(x2) + 6(x2)ti* + C{X2)X2)X3 dt — X2X^. 



By (2.2), (3.8) and using the property w ^ u"^ 1, there exists a constant C such that 

lk2llvFi>°° ^C, Vn e N. 



(3.10) 
(3.11) 



(3.12) 



(3.13) 



In (3.13) and until the end of the proof of Theorem [2| we denote by C different positive constants 
which are independent of n. It follows from the Arzela-Ascoli theorem and (3.13) that there exists 
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X2 S C^{[tQ,ti\) and a subsequence {x^'-)i,z^ such that x^^ — )• X2 in CO([to,ti]) as / +00. Now, 
we choose the corresponding subsequence and again, we write instead of (u"');^^ 

and (a:;2)nGN instead of {x^^)i^f>]. We have 



n" ^ n* in L°^(to,ti) as n — )• +00, 
X2 — )• X2 in C''([to,ti]) as n — )• +cxd. 



By solving (3.8), we have 

^2W=^2+ / (a(x^) + 6(x^K)ds, Vt G [to,ti]. 



(3.14) 
(3.15) 



(3.16) 



to 



Using (3.14) and (3.15) and letting n — )• +00 in (3.16), we obtain 

rt 



X2{t) = X2+ I [a{x2) + b{x2)u^)ds,Mt e [tQ,ti], 



which together with (3.10) shows that 

X2(t)=X2(t), VtG[to,tl]. 



From (3.15) and (3.17), we have 

X2 ^ X2 in C''([to, ii]) as n — )■ +00. 



Let us only treat the case where 



X2{tl) > Vs, 



(3.17) 
(3.18) 
(3.19) 



(the case X2{ti) = Us can be treated similarly and the case X2{ti) < t/s is simpler). By ( 3.19| ), there 
exists t £ [to) ^i) such that 



X2ii) = Vs- 



(3.20) 



Moreover, by (|2.6|), this t is unique. From (|3.18|) and (|3.19), we have, for n large enough, which 

(3.21) 



will be from now on always assumed, 

X2itl) > Vs- 

Hence, as for X2, there exists one and only one tn G [ioi^i) such that 

X2V-n) = Vs- 

Let us prove that 

t„ — 7- t as n — 7- +00. 



In order to prove (3.23), we may assume, without loss of generality, the existence of t G [to, ^i] such 
that 

t„ — 7- t as n — 7- +00. 



Letting n — )• +00 in (3.18), and using (3.22) together with (3.24), we get that 

X2{i) = Vs. 



(3.22) 

(3.23) 

1] such 

(3.24) 
(3.25) 
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Since t is characterized by (3.20), (3.23) follows from (3.24) and (3.25). 



From (2.3), (3.9) and (3.11), we have 



x^{t) = x^e'^^^*-*"), Vt G [to, in] and x^{t) = x[| e^^^ \ Vt G [in,h] 
X3(t) = x!|e^^(*-*«), Vt G [to,t] and X3(t) = x|] e'^^*"*"), Vt G [Mi]. 



From ( |3.23[ ), ( |3.26[ ) and ( |3.27[ ), we get that 

3^3—^2:3 in C^{[tQ,ti]) as n +00. 



Finally, combining (3.7), (3.12), (3.14), (3.18) and (3.28), we get 



lim J(n") = J(n^,), 



which shows that is an optimal control. This concludes the proof of Theorem [2j 



(3.26) 
(3.27) 

(3.28) 



3.2 Pontryagin Maximum Principle 

In this section we prove a PMP for our optimal control problem. For sake of simplicity, we 
denote, from now on. 



p{x, u) := -(a(x2) + 6(^2) u) x^, q{x) := -Cg X2 X3, Vx = {xi,X2, X3)*'' G M^. 
Let us denote by x : M — t- M the characteristic function of (— oo,ys), i.e. 



1, '\/x2 G {-00, Vs), 

0, yx2 G [us, +00). 



Hence, the cost function J for our optimal control problem problem (2.13) is 

J{u)= / {p{x,u) + q{x)x{x2)) dt - X2xl. 

Jto 

Let us define the Hamiltonian 



n : 



{x,u,ip) = {{x ,x , . . . ,x ),u,{ij ,ip , . . . ,ip )) 1-^ 7i{x,u,'ilj) 



by 



N 



N 



n{x,u,i^) := J;(/(x^n),V') - Y.ip{x^u)+q{x'^)x{xl)). 



k=l 



k=l 



(3.29) 



(3.30) 



(3.31) 



(3.32) 



In (3.32) and in the following (a, b) denotes the usual scalar product of a G and b G M^. Let us 
also define the Hamilton- Pontryagin function H : (M'^)^ x (M'^)^ — M by 



H{x,ip) := max T-l{x,u,ip). 



(3.33) 



Our goal in this section is to prove the following theorem. 
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Theorem 4. Let u-t G L°°{{tQ,ti); [w,l]) be an optimal control for the optimal control problem 
(2.13). Let = {x^i,x^2j^*3y^j k = ,N, be the corresponding optimal trajectory, i.e. x'^i E 

W^'°°{to,ti), x^2 £ W^'°^ito,ti), 2:^3 G W^'^{to,ti) are solutions to the following Cauchy problems 



Xi , 



1, <i(io 



^2 ) 



(3.34) 
(3.35) 
(3.36) 



U Us £ {x^2i^)'^ * ^ [^Oi^i]}; ^A: £ [*0)^i] i/ie exii time for the k-th Dirac mass, i.e. the 
unique time ik G [^0,^1] such that x^2{'tk) = Us- If Us 4- {^*2(*); (^0,^1]}; ^ei tfc = ti + 1. 
T/ien, i/iere exists vector functions ijj^ = (t/'J^, V'f > V't)*''? s^ic/i that ipi G T^-^'°°(to, ti), V'2 ^ 
H^^'°°(((to,4) U (4,ii)) n (to,ti)) anc? Vt e W^^'°°(*o,ii) sfc/i i/iat 



0, 



= -(a'(x;^2) + b'{x':2)u,)ijl - {a{x^2) + 6'(x^2)^^*)43 

-c,x^3X(42) ((to,4)U(4,ii))n(to,ii), 

i^iih) = i^lih) = 0, 



(3.37) 

(3.38) 
(3.39) 
(3.40) 



and 



• ifik < h, 

^-1(4+0) -V'2(4-0) G 

• «/4 = 



cs a;J3(4)(ys + ^^1(4)) c, x^3(4)(y. + V't(4)) 



«(ys) + fe(ys) 



0, 



-V'f (ii) e 



0, 



(3.41) 
(3.42) 

(3.43) 



a{ys) + b{ys)w- 
Moreover the following condition holds 

'H{xl{t),u^{t),i;^{t)) = H{xl{t),i)^{t)) for almost every t G (to,ti). (3.44) 

Proof of Theorem |4| For sake of simplicity, we give the proof only for one Dirac mass (A^ = 1) 
and, again, we omit the index k = 1. For more than one Dirac mass, the proof is similar. Our 
proof is inspired from j31j . 

Step 1. Let (wj)jgN* be a sequence of elements in C°°(M) such that 



^ 



and, for some C > 0, 



Wi{x)dx = l, support liJj C [—1/i, 0], Vi G N*, 



|u;■(x)KCi^ VxgM, ViGN* 



(3.45) 



(3.46) 
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(clearly such a sequence does exist). Then, we define a sequence of functions {xi)ieN* from M into 
M as follows: 



Xiy) Wi{x - y) dy 



Wi{x - y) dy 



Wi{z) dz, Vi G N*, Vx G M. (3.47) 



One easily sees that the functions Xi thus defined possess the following properties (compare to [31} 
Lemma 1 and Lemma 2]): 



0^Xi{x2)^x{x2), ViGN*, Vx2GM, 
Xi{x2) X{x2) as i oo, \/x2 G M, 
0^Xi(2;2)^l, ViGN*, Vx2GM, 
Xi = 1 in (-00, ys - (1/i)] and Xi = in [ys, +oo), Vf G N*. 



(3.48) 
(3.49) 
(3.50) 
(3.51) 



Let u* be an optimal control for the optimal control problem (2.13) and let be the associated 
trajectory. Let (zj)i6N* be a sequence of elements of C^{[to,ti]) such that the following conditions 
hold: 



Zi —7- u^, in L^(to,ti) as i — +oo, 
sup \zi{t)\ ^2, i = 1,2,-- - . 



(3.52) 
(3.53) 



It is again obvious that such a sequence of functions (zi)jgN* does exist. 
Let us first prove the following lemma which will be used later. 

Lemma 5. There exists C > such that, for every u G L°°((to,ii); [u^jl]) o.nd every X2 G [0, y^]; 
the following holds 

{X{x2{t))-Xi{x2mdt^-, VfGN*, (3.54) 
to ' 

where X2 is the solution to the following Cauchy problem: 

X2 = a{x2) + b{x2) U, X2{to) = X2. 

Proof of Lemma [sj The case where 2:2(^1) < ys is trivial, we treat the case where ^2(^1) > ys- 
Then, there exists one and only one t G [to,ti) and one and only one ti G [to,ti) depending on i 
such that 

X2{i) = ys, X2{ti) =m.Siy.{ys- {l/i), X2). (3.55) 



Using (3.48) to (3.51), we obtain 



to 



By (3.55 ), we have 



{X{x2{t)) - Xi{x2{t))) dt = / {x{x2{t))-Xi{x2{t)))dti;:i-ti 



(3.56) 



xl + 



to 



(a(x2) + b{x2)u) dt, 
max ( ys - (l/i), X2) = X2+ / (0(^2) + b{x2)u) dt, 



to 
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which together give 

/■* 1 

/ {a{x2) + b{x2)u) dt ^ -. (3.57) 
By (2.6), there exists o" > such that 

a{x2) + b{x2) u ^ a, Vx2 G [0, y^], Vu G [w, 1]. (3.58) 

From (|3.57|), by choosing C = — , we get 

C 

0<i-ti^-, (3.59) 
t 

which, together with (3.56) concludes the proof of Lemma [5] ■ 
Let /i : X M be defined by 

fi{x,u) := ( a{x2) + b{x2)u\ , Vx = (xi, X2, X3)*' G M^, Vu G M. (3.60) 
\ CsXiix2)x3 J 

Let us also define Ji : L°^{{to, ti); [w, 1]) M by 

Ji{u):= r {p{x,u)+q{x)x^{x2))dt+ ^ \u{t) - Zi{t)\'^ dt - x^xl, (3.61) 
J to V 1 J to 

where x : [to, ^i] is the solution to the Cauchy problem 

X = fi{x,u), x{to) = x^. (3.62) 

We consider the following optimal control problem 

minimize Ji{u) for u G L°°((to,ti); ["U^, 1])- i'Pi) 

For any i = 1, 2, • • • , problem (Vi) is a "smooth" optimal control problem. By a classical result in 
optimal control theory (see, e.g., 122[ Corollary 2, p. 262]), there exists an optimal control Uj for 
problem (Vi). Let Xi be the optimal trajectory corresponding to the control Ui for dynamics (3.62). 
We have the following lemma. 

Lemma 6. The following holds as i ^ +00 

Ui ^ in L'^{to,ti), (3.63) 
Xi x^ in C°([to,ti];M^), (3.64) 
Xiixi2) ^ Xix*2) inL^{to,ti). (3.65) 

Proof of Lemma [6| Since Uj is an optimal control for the optimal control problem (Vi) and 
ti* G L'^{{tQ,ti); [w, 1]) is an admissible control for this problem, we have 

Mui) ^ JiM, yi G N*. 
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By (pTl), 



{p{xi{t),Ui{t)) + qixi{t)) Xiixi2{t))) dt+— \ui{t) - dt 

I to yiJto 

^ / {pix4t),u4t))+q{x4t))xi{xMt)))dt+ ^ / \u4t) - Zi{t)\'^ dt, (3.66) 
J to V i J to 



where Xi = {xii,Xi2,Xis) and x^ = {x^,i,x^,2,x^.3) are solutions to the Cauchy problems 

Xi — fiixii^i) , Xj (to) — X ) 
X* = /i(x*,M*), x*(to)=2;°. 



(3.67) 
(3.68) 



Since u* is an optimal control for the optimal control problem (2.13) and m is an admissible control 
for this problem, we have, for every z G N*, 

' {p{x,{t),u,{t)) + q{x,{t)) xMt))) dti^ {p{x^{t),u^{t)) + q{x^{t)) x{x^2{t))) dt, (3.69) 
to J to 



where x=k = (x=ki, x*2, 2^*3)*'^ and Xj = {xii,Xi2,Xi3) are solutions to the Cauchy problems 

X^ — f{^^}'U^^, x^(to) — ^ ? 
Xi = f{Xi,Ui), Xi{to)=X^. 



By (3.48) and note that q ^0, the inequality 

rti 



q{xi{t)) x{xi2{t)) dt / q{xi{t)) Xi{xi2{t)) dt 
to J to 



ti 



(3.70) 
(3.71) 



(3.72) 



holds for every z G N*. Combining (3.69) and (3.72), we obtain 

r-ii 



to 



(p(x*(t),u^,(t)) + q{x^{t)) x{x*2{t))) dt ^ 



to 



{pixi{t),Ui{t)) + q{xi{t)) XiiS:i2{t))) dt. (3.73) 



By (3.66) and (3.73), we have 



/ ' \Ui{t) - Zi{t)\'^ dt 

yt Jto 

rti rti 
^ / {pix^{t),u^{t)) + q{x^{t))xiix*2it)))dt- {p{x^{t),u^{t)) + q{x^{t))x{x^2it))) dt 

Jto Jto 

rti rti 
+ / {p{xi{t),Ui{t)) + q{xi{t))xi{xi2{t))) dt - j {p{xi{t),Ui{t)) + q{xi{t))xi{xi2{t))) dt 
Jto Jto 

+ ^ r \u,{t) - Zi{t)\'^ dt. (3.74) 
vi Jto 



By definition of / and fi in (3.3) and (3.60), we have 



Xi2{t) = Xi2{t) and x*2(t) = x^2{t)-, Vt G [to,ti]. 



(3.75) 
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Using ([3^, dslslj), KT^ and Kf^, one has 



tl 



Ui{t) - Zi{t)\Ut i^C \x*3{t) - X*3{t)\ dt + C \ii3{t) - Xi3{t)\ dt 



to 



tl 



to 



+ c 



tl 



{x{x*2{t)) - Xi{x*2{t))) dt 



to 



+ 



tl 



'I Jto 



\u4t) - Zi{t)\^dt. 



(3.76) 



In (3.76) and until the end of the proof of Theor em [4| we denote by C different positive constants 
which are independent of i. From (3.48), (3.68), (3.70) and (3.75), we have 



' |x,3(t) - X,3(t)| dt = r \xt e^*o -^Xd^*^(s))ds _ ^0 c.x(-.2(s))d.| 



to 



dt 



to 



^ C 



to ''to 



{x{x*2{s)) - Xi{x*2{s))) dsdt. 



Applying Lemma [5] to (3.77), we get 



tl Q 

\x*3{t) - X*3(t)| dt ^ — . 



to 



Similarly, we can prove 



tl (J 

\xi3{t) - Xi3{t)\ dt ^ — . 



to 



Combining (3.76), (3.78) and (3.79), and noticing (3.52), we get 



— dt — )• as i — )• +CXD. 



to 



From ( |3.52D and ( |3.80D , we get that 

Iki - ■"*llL2(to,ti) as i — ;> +00. 



Thus, property (3.63) is proved. 



Let us now prove (3.64). Using in particular (3.63), one easily sees that 

Xii{t) = x^i{t), ViGN*, VtG[to,ti], 
Xi2 ^ x^,2 in C°([to,ti]) as i — ^ +oo. 



Moreover, the same proof as that of (3.28) shows that 

Xi3^x^3 in C°([to,ti]) as z +oo. 
Finally, let us prove (3.65). From (3.51) and ( 3.83| ), 

{x*2{t) + Vs) => (^.ji™. Xi{Xi2{t)) = x{x*2{t))] , Vt G [to,ti] 
But, as we have already seen in section 



(3.77) 



(3.78) 



(3.79) 



(3.80) 



(3.81) 



(3.82) 
(3.83) 



(3.84) 



3.1 



there exists at most one t G [to, ti] such that x^:2{t) = Us- 
Property (3.65) follows from ( |3.48[ ), (3.84) and Lebesgue's dominated convergence theorem. This 
concludes the proof of Lemma [6j ■ 
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Step 2. We now deduce necessary optimality conditions for the optimal control problem (2.13) 
in the form of PMP. Suppose that Xi and Ui is an optimal pair for problem (Vi). The Hamiltonian 
and the Hamilton- Pontryagin function for problem (Vi) are respectively 



ni{t,x,u,'ilj) = {fi{x,u),il)) - {p{x,u) + q{x)xi{x2)) - - Zi{t)\^, 
Hi{t,x,ip) = max X, 

m£[w,1] 



(3.85) 
(3.86) 



By the PMP -see, e.g., [22l Theorem 2, p. 319] or ^ Section 6.5]-, there exists an absolutely 
continuous function tpi : [to,ti\ — )• such that 



dfi 1 d 

—^{xi{t),Ui{t)) i)i + — (p(xi,n) + q{xi)xi{xi2)) 
ox J ox 



and 



7ii{t,Xi{t),Ui{t),ilji{t)) = Hi{t,Xi{t),^i{t)) for almost every t € (tc^i)- 
Let us denote ipi = (V'ii) V'j2) ^is)*'^- From (3.29), (3.60), ( |3.87 ) and (3.88), we have 

i^ii = 0, 

i^i2 = -{a'{Xi2) + h'{xi2) Ui) ■lpi2 - Cs Xi(Xi2) Xi-i Tpis - (a {Xi2) + b'{Xi2) Ui) Xi2, 

- Cs Xi{Xi2) Xi3 - Cs Xi2x!i{Xi2) Xi3, 
fpiS = -Cs Xiixi2) 1pi3 - ia{xi2) + b{xi2)ui) - Cs Xi2 Xi{xi2), 
Ipilih) = 1pi2{ti) = Tpisih) = 0. 



Let ipi : [to, ti] — M be defined by 



From ( |3.90[ ), ( |3.93[ ) and ( |3.94[ ), one has 

iPiiit) =Mt), VtG[io,ti]. 
Let ip3 G T^^'°°(to, ^i) be the solution to the following Cauchy problem: 

"03 = -CsX{x*2) tp3 - ia{x^2) + b{x*2)u*) - CsX^2 X{x*2), 1p3{tl) = 0. 

From ( [3^ , ( [3^ , ( [3^ , ( [3^ , ( [3^ and ( |3^ , one easily gets that 

V'is "03 in C°([to,ti]) as i - 
We now deal with -042- By (3.45) and ( 3.47| ), we have 

support Xi C [vs - {l/i),ys]. 



-oo. 



(3.87) 
(3.88) 

(3.89) 

(3.90) 

(3.91) 
(3.92) 
(3.93) 

(3.94) 

(3.95) 

(3.96) 

(3.97) 

(3.98) 



Moreover, from (3.45), (3.46) and (3.47), we obtain 



\Xi{x2)\ Ci, \/x2 € [Vs - {l/i),ys 



(3.99) 



18 



Theorem |4] in the case where x^:2{to) = X2 = Us or x^:2{ti) < yg follows directly from the standard 
PMP. Hence, we may assume that 



X*2{tQ) <ys X^2itl)- 

Let us first treat the case where 

x*2{to) <ys < x^2{ti). 
Then, again, there exists one and only one t G (to)^i) such that 

x*2{i) = Vs- 



(3.100) 



(3.101) 



(3.102) 



always be assumed, there exists one and only one ti G (to)^i) and one and only one ti G (to)*i) 
such that 

Xi2{ii) = Vs, Xi2{ti) = ys- (1/0- (3.103) 



Using (3.63) and (3.64), and proceeding as in the proof of (3.23), one has 

tj — >• t and — )• t as i — )• +00. 



From (3.99) and (3.103), we get 



lx-(xi2(t))K ci, ytG[ti,u] 



From (3.98) and (3.103), we have 

Xi{xi2{t)) = 0, t€[to,ti]\[uJi 



Proceeding as in the proof of Lemma [Sj one has 



< t; - U ^ 



C 



From ( |3.64D , ( |3.9lD , ( |3.93D , ( |3.97D , ( |3.105D , ( |3.106D and ( |3.107D , we get 

\\'>Pi2\\L--({to,hMk,h)) ^ C, 



fi2||L-'(to,ti) 



^ c. 



(3.104) 
(3.105) 
(3.106) 

(3.107) 



(3.108) 
(3.109) 



From ( |3.108 ) and (3.109), and extracting if necessary a subsequence, we get the existence of ip2 G 
VF^'°°((to,0 U (t,ti)), such that, as i +00, 

ipi2^^2 m C\[to,i-e]U[i + e,ti]), Ve > 0. (3.110) 

Letting i ^ +00 in ( |3.9lD , one gets, using ( |3.63D , ( |3.64D , ( |3.98D , ( |3.106D and ( |3.110D , 

■02 = -(a'(x*2) + &'(a;*2)w*)'i/'2 - {a{x^2) + b' {x^.2)u*)x*3 

-CsX^3Xix*2) in ito,ti)\{t}. (3.111) 



Moreover, from (3.93) and (3.110), we have 



Mh) = 0. 



(3.112) 
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We now prove the gap condition ( 3.41[ ). Let us integrate (3.91) from ti to ii, we get 

i^^2iii) - i^i2{ii) = A{i) + B{i), (3.113) 

with 



A{i) :- 



{{a'{xi2) + b'{xi2) Ui) {'lpi2 + Xis) + Cs Xiixi2) Xis) dt, 



Bit) 



Cs Xi3 {xi2 + -fAis) Xi(a^i2) dt. 



From (3.64), (3.104), (3.108), (3.109) and (3.114), one gets that 



A{i) — )- as i — )- +00. 



For B[i), we perform the change of variable r = Xi2{t). Using (3.103) and (3.115), we get 



B{i) 



Cs Xi3{x^^{T)){T + i}i3{x^^{T))) 

— j — Xiv) dr. 

ys-{l/i) a{T) + b{T)u{x^^ (t)) 



Let us point out that, from (3.45), (3.47) and (3.51), one has 



ys-{i/i) 



(3.114) 
(3.115) 

(3.116) 

(3.117) 

(3.118) 



From (3.58), (3.64), (3.97), (3.117), (3.118), one gets that 



Csx.sii)iys+ Mi)) ^ liminf S(z) ^ hmsupS(.) ^ ^-3(t )(^. + V^3(^^)) ^ 
a{ys) + o{ys) i^+oo i-^+oo a{ys) + b{ys)w 



which, together with (3.104), (3.108), (3.110), (3.113) and (3.116), gives (3.41). 



Let us now treat the case where 

x*2{ti) = ys- 

One adapts the proof given above in the fohowing way. 

• If, for some i — )• +oo, Xi2{ti) ^ ys — (1/*), then one gets ip2{ti) = 0. 

• If, for some i — )• +oo, Xi2{ti) > ys, then, proceeding as above, one gets 

CsX^3{i)ys CsX^3{i)ys 



(3.119) 



aiVs) + b{ys) ' a{ys) + b{ys) w 



(3.120) 



If, for some i — )■ +oo, ys — (1/i) < Xi2{ti) ^ ys, one just need to replace ti by ti in the above 



proof. Then, one replaces (3.118) by 



/ x^{r)dT^-l, Xi^O, 

Jys-{l/i) 



and one gets (3.120). 
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Let us now prove the maximum condition (3.89). By (3.63), extracting if necessary a subse- 



quence of the Uj's and still denoting by (Mi)igN* the extracted sequence, there exists a Borel subset 
A/q of {to ,ti) of Lebesgue measure such that 



lim Ui{t) = u^,{t), Vt G (^0, ii) \ A/q. 



(3.121) 



Let i € N*. By (3.89), there exists a Borel subset A/i of {to,ti) of Lebesgue measure such that 

ni{t,Xi{t),Ui{t),iPi{t)) = Hi{t,Xi{t),iPi{t)), Vt G {to,h)\Mi. (3.122) 



Let 



M:={i}[J (UieN*A/'i) 



Then is a Borel subset of {to,ti) of Lebesgue measure 0. Let u £ [w, 1]. By (3.122), 

ni{t,xi{t),ui{t),^i(t));?ni{t,xi{t),u,iPi{t)), vieN*, vt g (io,ti) \aa. (3.123) 



(3.97), (3.110) and (3.121), one gets that 



Let V := (V'i,V'2,V'3)*'- Letting w +00 in (3.123) and using (3.53), (3.64), (3.84), (3.85), (3.95) 



'H{x^{t),u^{t),i;{t)) ^ n{x^{t),u,il^{t)), VtG {to,ti)\Af, 
which gives (3.44). This concludes the proof of Theorem |4j 



3.3 Proof of Theorem H 

In this section, we use the necessary optimality conditions given in Theorem [4] in order to prove 
Theorem [sj From now on, we assume that the target time ti satisfies ti > to so that all the cells 
will exit from Phase 1 into Phase 3 before time ti. We give a proof of Theorem [3] in the case where 
A'^ = 1 in section 3.3.1 In section [3.3.3 we study the case where > 1; in this case we need 
additionally to analyze the dynamics between different exit times tfc, k = 1,2, ••• ,N, to obtain 
that there exists one and only one switching time and that the optimal switching direction is from 
u = w to u = 1. In both cases = 1 or > 1, we give some numerical illustrations, respectively 
in section 13.3.21 and section 13.3.41 

3.3.1 Proof of Theorem H in the case A^ = 1 



The Hamiltonian (3.32) becomes 



T-L{x, U, V") = V'l + {a{x2) + b{x2)u)'4)2 + c{x2)x^ll)^ + (0(^2) + b{x2)u)x^ + c{x2)X2X'i 

= {a{x2) + c{x2)x2) X3 + -01 + a{x2)i^2 + c(x2)x3 i/'s + h{x2){x^ + V'2) u. (3.124) 



Let u be an optimal control for the optimal control problem (2.13) and let x = {xi,X2,x^Y^ be the 



corresponding trajectory. Note that, by (2.2), 6(^2) > 0. Then, by (3.33), (3.44) and (3.124), one 
has, for almost every t G (to 5*1)) 



U{t) = l if X3(t) +02(t) > 0, 
u{t)=W if X3(t) + V2(t) < 0. 



(3.125) 
(3.126) 
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Let us recall that, under assumption (2.14) of Theorem [3| there exists one and only one i G [toj^i) 
such that 



Then 



X2{i) = Vs. 



We study the case where t > tQ, the case t = to being obvious. Thanks to (3.41), we get 



(x3 + V'2)(i + 0) - ix3 + ip2)ii-0) ^ Csxsii) 



aiVs) + Kvs 



By (3.35) and (3.39), we get 



dix2 + ips) 
dt 



-{X2 + V's) CsX{x2) 



and then, using also ( |3.30D , ( |3.40D , ( |3.127D and ( |3.128D , we obtain 

Z/s + V'sC* ) = (a:2 + V'3)(i ) = (2:2 + V'3)(ii) ^ 



Combining (3.129) with (3.131), we get 



(2;3+'02)(t-O) ^ (x3 + ^2)(t + 0) -C,X3(t) 



When t / t, from (2.3), (3.30), (3.36) and (3.38), we obtain 



^(3:^3 + V'2) 
dt 



-{a{x2) + h'{x2)u) {x^ + ^2). 



Hence, using also (2.2), we have 

(x3 + V2)(i + 0) = (x3+V2)(tl)e-^^(2"^(^)-^^"(^))"^ 

Using the first inequality of ( [2l5| ), ( |3l2| ), ( |3l28l ) and ( [3T341 ), we get 

(x3+V'2)(t + 0) ^ X3(ti). 

Noticing that X3(£ + 0) = X2,{ti) and using ( 3.132[ ) and (3.135), we get 

(x3+^2)(t-0)^.3(tO(l-C. ^(^/;^(^^^ ). 



From the second inequality of (2.15) and (3.136), we get 

(x3 + V'2)(i-0) <0. 



which, together with (3.133), gives us 

(X3 + V'2)(t) < 0, tG[to,i). 



Moreover, by (3.42), we have 



(X3 + V'2)(il) =X3(tl) >0, 



which together with (3.133), gives 



(X3+V'2)(t) > 0, tG(t,tl]. 



(3.139) 



Taking t* = i and combining (3.138) and (3.139), with (3.125) and (3.126), we conclude the proof 
of Theorem [3] in the case where = 1. ■ 
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3.3.2 Numerical illustration in the case = 1 



For one Dirac mass, the optimal switching time is unique. Assumption (2.15) is not necessary 
to guarantee that the optimal control is a bang-bang control with only one switching time. It 
is only used to guarantee that the optimal switching time coincides with the exit time. We give 
a numerical example to show that when c<j is "small", there is no switch at all and the optimal 
control is constant {u = 1), while when Cg is "large", there is a switch occuring at the exit time 
(see figure [s]) . 

The default parameter values are specified in Table [T] for the numerical studies. 



to 


initial time 


0.0 


h 


final time 


17.0 


Cl 


slope in the b{y) function 


11.892 


C2 


origin ordinate in the b(y) function 


2.288 


Vs 


threshold maturity 


6.0 


w 


minimal bound of the control 


0.5 



Table 1: Default parameter values 




cs 



Figure 3: Value of the cost function J with respect to the switching time (t) and Cg parameter 
in the case of one Dirac mass. When Cs is "small", there is no switching time {t = 0) and the 
optimal control is constant (u = 1), while, when is "large", the optimal control strategy consists 
in switching from u = w to u=l at a time coinciding with the exit time. The initial values are 
specified in the insert. 
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3.3.3 Proof of Theorem [3] in the case > 1 



Now, the Hamiltonian (3.32) becomes 

TV 



u, ^) = Y,{i<4) + c{xl)xl)x\ + ^\ + a(x^)V^f + c{xl)x\^l + b{xl){xl + . (3.140) 



fc=i 



Reordering if necessary the x s, we may assume, without loss of generahty, that 



10 ^ ^20 ko ^ ^ NO 



(3.141) 



Let u be an optimal control for the optimal control problem (2.13) and let x = {x^ 
with 3) , be the corresponding trajectory. From (|3.141l), we have 

tN < tN-1 < ■ ■ - tk < . . . < ti. 

Let <I> : [to,ti] M be defined by 

TV 

m ■.= Y.biximxi{t)+4{t)). 



? • • • 1 *^ , 



X , . . . X 



(3.142) 



(3.143) 



k=l 



Noticing that b{x^) > 0, by < \3.33\) , < \3.U\) , ( |3.140D and ( |3.143D , one has, for almost every t € (to,ii), 



u = w, if <5(t) < 0, 
n = 1, if $(t) > 0. 



We take the time-derivative of (3.143) when t ^ tk, k = 1, ■ ■ ■ , N. From (2.2), we obtain 

TV 



^(i) = E(ci(^2)' + 2c2X2')(4 + V': 



(3.144) 
(3.145) 



(3.146) 



k=l 



Similarly to the above proof for one Dirac mass, we can prove that, under assumption (2.15), we 
have, for each k = 1, • • • , A^, 



{x^ + i'2){t) < 0, when te {to, ik), 
(4 + V'2)(*) > 0, when t G (4,ti). 

By ( |3l42l ), ( [3I43I ), ( [3I47I ) and ^J^, and note that 6(x|) > 0, we get 

$(t) < 0, when t G (to^Af), 
$(t) > 0, when t G (ti,ti). 



(3.147) 
(3.148) 



(3.149) 

(3.150) 



The key point now is to study the dynamics of $ between different exit times t^- Let A; G {1, • • • , A^— 
1} and let us assume that 

$(t) = 0, for some tG(4+i>4)- (3.151) 



From (3.143) and (3.151), we get 



;(i) + V'f(i) = -E 



{x\{t)+^\{t)). 



(3.152) 
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From (|3.147D and (|3.148D, for every t E (4+i,4), 



+ V2(i) < 0, when i k - 1, 
xi{t) + ip'2{t) > 0, when i ^ A; + 1. 



From ([2^, ( [3l46l ) and ([3l52j), we get 



^3 + '^2 ( 2i k 



^<k-l ^(^2) 



cjxix^ + 2c^ + ciC2{xl + x^)){xi-x^) 



From (3.141), we get 



+ Yl ^l^t'^ {44x^ + 2cl + ciC2{xl + x^)){xi - x\) 

x\{t) - x\{t) < 0, when i ^ k - 1, 
xi{t) - X2{t) > 0, when i k + 1. 



Using ( I3.153D to ( |3.157D , we get 

> whenever ^(t) =0, Vt G (4+1,4)- 



(3.153) 
(3.154) 



(3.155) 



(3.156) 
(3.157) 



(3.158) 



Combining ( |3.144| ), ( |3.145| ), ( |3.149| ), ( |3.150| ) and ( |3.158D together, we get the existence oft* E (to, ii) 
such that 

u^: = w in (to,i*) and n* = 1 in (t*,ti). 
This concludes the proof of Theorem [3} ■ 



3.3.4 Numerical illustration in the case > 1 

The optimal control is not unique for more than one Dirac mass. Let us consider the case of 
two Dirac masses as an example. The optimal switching time may happen either at the first exit 
time or at the second exit time (see figure [4]) , or between the two exit times (see figure Is]) . 
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1.5 



Figure 4: Value of the cost function J with respect to the switching time (t) in the case of two 
Dirac masses and a "large" value of Cg {cg =1.0). In the left panel, the three-part curve represents 
the value of the cost function obtained after switching from u = w to u = 1 aX, time t. Blue 
dashed curve: switching time occurring before the first exit time; green solid curve: switching 
time occurring in between the two exit times; red dashed curve: switching time occurring after the 
second exit time. The initial values are specified in the insert. The right panel is a zoom on the 
green solid curve displayed on the left panel. There are two optimal switching times which coincide 
with the two exit times. 




Figure 5: Value of the cost function J with respect to the switching time {t) in the case of two 
Dirac masses and a "large" value of (c^ =0.8). In the left panel, the three-part curve represents 
the value of the cost function obtained after switching from n = u) to n = 1 at time t. Blue 
dashed curve: switching time occurring before the first exit time; green solid curve: switching 
time occurring in between the two exit times; red dashed curve: switching time occurring after the 
second exit time. The initial values are specified in the insert. The right panel is a zoom on the 
green solid curve displayed on the left panel. There is one single optimal switching time, which 
occurs in between the two exit times. 
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4 Optimal control in the PDE case 



In this section, we study the optimal control in the PDE case. We give the proof of Theorem [1} 
We first give an explicit expression for the cost function J defined in (2.12). 
Let us define a new map 



by requiring 



e: [0,2/,] xL°°((to,ti);[tu,l]) ^ [t^M] 
{yo,u) ^ e{yo,u) 



(4.1) 



where ^ is defined by (2.19). Note that, under assumption (2.14), one has, for every yo £ [0,2/, 
the existence of t G [to, ti] such that 



^(t,yo,u) = ys- 



(4.2) 



Again, (3.58) implies that there exists at most one t E [toj^i] such that (4.2) holds. This shows 



that e is well defined. Moreover, proceeding as in the proof of (3.23) and using standard results on 



ordinary differential equations together with (3.58) gives the following lemma 



Lemma 7. Let {yQ )n&i be a sequence of elements in [0, ys] and (n")„gN a sequence of elements 
in L°°{{tQ,ti)]\w,l]). Let us assume that, for some yo ^ [0,2/s] and for somen ^ L°°{{tQ^ti)][w,V\), 

Vo ^yo as +oo, 

u in L°°(to,ti) as n ^ +oo. 



Then 



e{yQ,u"') — )■ e{yQ,u) as n ^ +oo. 



Let now po be a Borel measure on M x M such that ( |2.20[ ) and ( |2.2l[ ) hold. Using ( |2.22[ ), ( |2.12 ) 
becomes 

J{u) = - f[ ^{t,,yo,u)e^'<y'^'^Upo{xo,yo). (4.3) 

J J [0,l]x[0,ys] 

In order to emphasize the dependence of J on the initial data po, from now on we write J{pQ,u) 
for J{u). 

It is well known that there exists a sequence ((xq", j/q", AQ"))i^j^„^„gN of elements in [0,1] x 
[0, ys] X (0, +oo) such that, if 



2/0 ' 



i=l 



(4.4) 



then 



lim // ip{xo,yo) dpQ{xo,yo) 

"-^+°° J J[0,l]x[0,j/,] 



[0,l]x[0,ys 



ip{xo,yo)dpoixo,yo), G C°([0,1] x [0,y,]). (4.5) 
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From Theorem [2] and Theorem|3| there exists t" G [ioi *i] such that, if li" : [to, ti] — )• [w, 1] is defined 
by 



then 



u'i = wm [to,C) and < = 1 in 



(4.6) 



(4.7) 



Extracting a subsequence if necessary, we may assume without loss of generahty the existence of 
G [to, ti] such that 



hm C = t*. 

n— >'+oo 



Let us define u^: : [to^^i] — ^ ["W^) 1] by 

= tt; in [to,U) and = 1 in (t=„,ti]. 



Then, using (4.6), (4.8) and (4.9), one gets 



^'(ti,-,0 ^ ^{ti,-,u^) in C°([0,ys]) as n ^ +00. 



Moreover, from ( |4.6[ ), ( |4.8[ ) and ( |4.9[ ), one has 

^ If* in L°°(to, ii) as n — >• +00. 



From Lemma [7] and (4.11), one gets 

e(-, u") — )• e(-, ti*) in C°([0, y^]) as ?i — +00. 



(4.8) 

(4.9) 
(4.10) 
(4.11) 
(4.12) 



From (4.3), (4.5), (4.10) and (4.12) and a classical theorem on the weak topology (see, e.g., |5, (iv) 
of Proposition 3.13, p. 63]), one has 



J(po,u") J{po,u^) as n +00. 
Let now u G L°°{{to,ti); [w,l]). From Lemma [Tj (4.3) and (4.5), one gets 

J{pQ,u) — J{pq,u) as n — +00. 
Finally, letting n — )• +00 in (4.7) and using ( |4.13 ) together with (4.14), one has 

J{po,u*) ^ J{po,u), 
which concludes the proof of Theorem [TJ 



(4.13) 
(4.14) 
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